User computing device with personal agent program for recommending meeting a friend at a service location based on current location, travel direction, and calendar activity

ABSTRACT

A long-term personal agent program, executable as network service and/or on one or more user computing devices and related method for identifying opportunities and making recommendations on behalf of one or more users, are disclosed herein. In one example, the personal agent program includes a monitoring engine configured to monitor and interpret a user&#39;s activities over time with a plurality of sensing and logging methodologies according to user authorization, the use of statistical methods for learning to understand a user&#39;s goals and behavioral patterns from data, and the use of procedures for computing the expected value of information guiding sensing and logging in different contexts. The personal agent further may include a recommendation methodology configured to make suggestions and to take actions on behalf of the user, in the present moment as well as for future times, based on inferences about user goals and opportunities in the world.

BACKGROUND

Prior to the advent of wireless navigation devices, drivers on thefreeway often relied upon signs that notified them of services thatcould be accessed at the next freeway exit. This situation could lead toinadequate planning, including multiple intense conversations invehicles during the short interval between the notification by sign andopportunities to exit the highway efficiently, regarding whether thepassengers desired to exit the freeway to access the services. With theadvent of wireless navigation devices, a vehicle passenger may now inputa desired service (e.g., gas station, restaurant) while traveling in thevehicle along a route, and view a list of such service locations and adistance to each along the route.

As helpful as such wireless navigational devices may be, vehiclepassengers are tasked with requesting a list of services from thesedevices prior to receiving a list of results. To accomplish this task, apassenger has to be cognizant of the desire to utilize a service aheadof time, and has to take the time to input the service request into thenavigational device. This can result in many missed opportunities toutilize services that the user would otherwise have desired to use. Asone example, this realization may arise after driving by a freeway exit,and hearing a young child plaintively announce from the back seat that“I have to go to the bathroom,” only to see a sign indicating “Next Exit43 miles”. This is but one example of many in which systems thatnecessitate that users be cognizant of their own needs in order torequest information on nearby services, fail to deliver satisfactoryresults for the user.

SUMMARY

A personal agent program executable on a user computing device andrelated method for retrieving recommendations on behalf of a user aredisclosed herein. In one example, the personal agent program includes asetup module configured to receive a user authorization to monitor useractivity across a plurality of computer programs that are used by theuser on the user computing device and/or one or more other usercomputing devices. The setup module is also configured to receive one ormore user recommendation preferences indicating product or servicerecommendations that the user would like the personal agent program toretrieve from a recommendation server.

The personal agent program also includes a monitoring engine configuredto monitor the user activity with the plurality of computer programsaccording to the user authorization. The user activity includes adetected current location of the user, and the monitoring engine isconfigured to learn a behavioral pattern from the user activity. Thepersonal agent program further includes a recommendation engineconfigured to make an inference that a trigger condition for one or moreof the user recommendation preferences will arise, based on the detectedcurrent location of the user, the behavioral pattern of the user, andone or more contextual factors. The recommendation engine is configuredto send a request to the recommendation server for a recommendation fora target product or service according to the one or more userrecommendation preferences. The recommendation engine is also configuredto receive the recommendation from the recommendation server, and todisplay the recommendation on a display associated with the usercomputing device.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic view of one embodiment of a computer systemincluding a personal agent program executable on a user computing devicefor retrieving recommendations on behalf of a user.

FIG. 2 is a partial detail schematic view of the personal agent programof FIG. 1, illustrating the manner in which inferences are made andrecommendation requests are generated.

FIG. 3 is a schematic view of a first example screen of therecommendation graphical user interface shown in FIG. 1.

FIG. 4 is a schematic view of a second example screen of therecommendation graphical user interface shown in FIG. 1.

FIG. 5 is a schematic diagram illustrating an example use case of thecomputer system including a personal agent program of FIG. 1.

FIG. 6 is a diagram illustrating one embodiment of a method forretrieving recommendations on behalf of a user.

FIG. 7 is a continuation of the diagram of FIG. 6.

DETAILED DESCRIPTION

FIG. 1 illustrates generally one embodiment of a computing system 5including a personal agent program 10 executable on a user computingdevice 12 for retrieving recommendations on behalf of a user 14. Asdescribed in more detail below, the personal agent program 10 includes asetup module 20, a monitoring engine 22, and a recommendation engine 24.

In one example, the user computing device 12 includes mass storage 28,memory 30, a display 32, a processor 34, and a location-awaretechnology, such as a GPS receiver 36. The GPS receiver 36 determinesthe location of the user computing device 12 based on the reception ofsatellite signals. Mass storage 28 may include the personal agentprogram 10 and a variety of other application programs, such as an emailprogram. 40, a calendar program 42, a telephone/messaging program 44, amobile device tracking program 46, and a browser 48. These programs maybe executed by the processor 34 using memory 30, with output displayedon display 32, to achieve the various functions described herein. Inother examples, user computing device 12 may include other componentsnot shown in FIG. 1, such as user input devices including touch screens,keyboards, mice, game controllers, cameras, and/or microphones, forexample. Further, although not shown in FIG. 1, it will be appreciatedthat other user computing devices 52 and 54 have similar components thatfunction in a similar manner as described above for user computingdevice 12.

It will also be appreciated, as described in more detail below, thatrecommendations may be retrieved, user activity may be monitored, andactions may be taken on behalf of a user 14 across multiple usercomputing devices, such as devices 12, 52 and 54, in adevice-independent manner. It will further be appreciated that this andother functionality, described below with respect to the personal agentprogram 10, may be executed and/or coordinated by a network-accessibleservice in communication with the multiple computing devices. Such aservice, may, for example, provide standard sensing and interactioninterfaces that enable linking and/or communicating with multiple usercomputing devices. Alternatively or in addition, these devices mayexecute agent software that provides communication protocols formonitoring user activity, making recommendations and taking actions onbehalf of a user.

The process by which the personal agent program 10 retrievesrecommendations on behalf of a user will now be described. In oneexample, the setup module 20 is configured to receive a userauthorization 56 from user 14, via a recommendation graphical userinterface (e.g., GUI) 82 displayed on the display 32 of the usercomputing device 12. The user authorization 56 authorizes the personalagent program 10 to monitor user activity across a plurality of computerprograms used by the user on the user computing device 12 and/or one ormore other user computing devices, such as user computing device 52 anduser computing device 54. An example of a computer program used on theuser computing device 52 may be a map program 58. An example of acomputer program used on the computing device 54 may be a socialnetworking program 60. It will be appreciated that the use of theseprograms on user computing devices 52, 54 is merely an example, andthese programs 58, 60 may instead or in addition be used on usercomputing device 12. Further, the various computer programs identifiedabove, including but not limited to email program 40, calendar program42, telephone/messaging program 44, mobile device tracking program 46,and browser 48, may be used by the user on the computing device 12,and/or other user computing devices 52, 54.

The monitoring engine 22 is configured to monitor the user activity withthe plurality of computer programs according to the user authorization56. In FIG. 1, user activity in the email program 40 is indicated bydashed line 40′, user activity in the calendar program 42 is indicatedby dashed line 42′, user activity across the telephone/messaging program44 is indicated by dashed line 44′, user activity in the mobile devicelocation tracking program 46 is indicated by dashed line 46′, useractivity in the browser 48 is indicated by dashed line 48′, useractivity in the map program 58 is indicated by dashed line 58′, and useractivity in the social networking program 60 is indicated by dashed line60′. The user activity may include a detected current location of theuser 14, which may be detected by the mobile device location trackingprogram 46 using GPS or other suitable tracking technologies.

Upon receiving the appropriate authorizations, the monitoring engine 22may also be configured to monitor friend activity 57 in various programsexecuted on a friend computing device 55 used by a friend 15 of the user14. Although not shown in FIG. 1, it will be appreciated that the friendcomputing device 55 may have similar components and programs thatfunction in a similar manner as described above for user computingdevices 12, 52 and 54. For example, the monitoring engine 22 may monitorfriend activity 57 in a calendar program, a mobile device locationtracking program and a social networking program that are executed onthe friend computing device 55.

In one example that is described in more detail below, the monitoringengine 22 may monitor user activity 46′ in the mobile device locationtracking program 46 of the user computing device 12 and friend activity57 in a mobile device tracking program in friend device 55 to determinethat the user 14 and friend 15 are traveling in directional trajectoriesthat will intersect at an intersecting location. The monitoring engine22 may also monitor user activity 42′ in the calendar program 42 of theuser computing device 12 and friend activity 57 in a calendar program infriend device 55 to check for availability of the user 14 and friend 15within a time period that includes an approximate time that the user 14and friend 15 may arrive at the intersecting location.

If the user 14 and friend 15 are available during the time period, thenthe recommendation engine 24 is configured to request and receive from arecommendation server 66 a recommendation 70 for a target service thatis provided within a threshold distance of the intersecting location.The recommendation 70 may then be displayed on display 32 of the usercomputing device 12. Additional description of the operation of therecommendation engine 24 is provided below. It will also be appreciatedthat, upon receiving the appropriate authorizations, the monitoringengine 22 may also monitor activity on computing devices that areassociated with other friends, relatives, colleagues and/oracquaintances of the user 14.

The monitoring engine 22 may also be configured to dynamically monitoruser activity by selectively activating or accessing at least one of theplurality of computer programs based on computing an expected value ofinformation that may be gleaned from the user activity. In this manner,the personal agent program 10 may gain access to additional data relatedto user activity, and thereby enhance the program's real-time decisionmaking capabilities and/or long-term data collection for learningpredictive models, as described in more detail below.

In one example, user activity from one or more of the computer programsmay be normally inaccessible due to user privacy preferences. Where thepersonal agent program 10 has received necessary user opt-inpermissions, the monitoring engine 22 may selectively access thesecomputer programs and monitor the user activity that otherwise would beunavailable. In another example, one or more of the computer programsmay be inactive for resource considerations, such as a deactivated GPSreceiver 36 for reduced power consumption, in a situation or at a timethat data monitoring would otherwise be useful. Again, provided that thenecessary permissions have been received, the monitoring engine 22 mayselectively activate the inactive program and monitor the user activityrelated to that program.

The monitoring engine 22 may selectively activate or access one of theplurality of computer programs at particular times and/or locations thatwill likely generate useful user activity information. The monitoringengine 22 may also determine when to selectively activate or access oneof the plurality of computer programs by computing an expected value ofthe information that may be gleaned from the user activity associatedwith the computer program. In one example, computing an expected valueof the information may include using heuristic procedures or otherexperienced-based evaluations. If the resulting expected value exceeds athreshold value, then the monitoring engine 22 may selectively activateor access the otherwise inaccessible or inactive computer program.

The monitoring engine 22 may also be configured to learn a behavioralpattern 74 from the user activity in the various computer programs onthe various user computing devices. In one example, the monitoringengine 22 may monitor the GPS receiver 36 to infer a user's response, orlack thereof, to a recommendation provided to the user. In one case, themonitoring engine 22 may determine that the GPS signal was lost at aparticular location, due to, for example, the user entering a parkinggarage or turning off the computing device. If the user has justreceived a recommendation for a product or service provided at thislocation, then it may be inferred that the user has responded favorablyto the recommendation.

Additionally, where the GPS receiver is later activated at the samelocation after a period of time, it may also be inferred that the userhas been present at that location for the period of time. Thisinformation may also be used to infer user satiation. In one example, aGPS location indicates that the user has stopped at a restaurant for anamount of time, such as 1.5 hours, that indicates that a meal was likelyconsumed. Using this data, the recommendation engine 24 may infer howlong it will be until the user may desire another meal.

While these behavioral patterns are learned from observation of activityof a particular user over time, the patterns for any particular user maybe based upon and compared to aggregate behavioral patterns generatedfrom observing an entire user population over time. Thus, the monitoringengine 22 may in one mode, learn user behavioral patterns by receivingaggregate behavioral patterns from the recommendation server 66, andexamining the user activity in the various computer programs for useractivity that matches the aggregate behavioral patterns within athreshold degree.

The personal agent program 10 utilizes user action histories andrecommendation preferences to determine what sorts of recommendations auser might like to receive. These user recommendation preferences may beimplicit, such as preferences inferred by the personal agent programitself, as indicated at 62, or may be explicit, such as preferencesinputted by the user, as indicated at 64. To that end, the monitoringengine 22 may be configured to create an inferred user recommendationpreference 62 based on the behavioral pattern 74, with the userrecommendation preference 62 indicating product or servicerecommendations that the user would like the personal agent program 10to retrieve from a recommendation server 66, as inferred by themonitoring engine 22. The setup module 20 may also be configured toreceive one or more inputted user recommendation preferences 64 from theuser 14 via user input into recommendation GUI 82.

In one use case example relating to an inferred user recommendationpreference 62, the monitoring engine 22 may observe a behavior pattern74 that includes user activity across a social networking program, suchas social networking program 60 on user computing device 54. Byobserving social interactions in which the user engages, the monitoringengine 22 may infer that one or more members of the user's social graphinfluences certain user purchasing decisions. For example, themonitoring engine 22 may observe that the user has dined at threerestaurants after receiving positive comments regarding each of therestaurants from a friend A in the user's social graph. Using thisinformation, the monitoring engine 22 may create an inferred userrecommendation preference 62 for restaurants that the user's friend Aprefers or has frequented. Information regarding the restaurants thatfriend A prefers or has frequented may also be gathered, for example, byobserving the user's activity across the social networking program,including communications involving friend A.

Additionally, the setup module 20 may be configured to receive a userprivacy setting 68 from user 14 indicating a category of data that theuser authorizes the personal agent program to examine. The privacysetting may also indicate whether sharing of the data is allowed with anoutside server, such as recommendation server 66. Specifically, the userprivacy setting 68 may indicate an examine-only category of the useractivity that the user authorizes the personal agent program 10 toexamine from the plurality of computer programs but not shareexternally, and/or a sharing-authorized category of the user activitythat the user authorizes the personal agent program to send to therecommendation server 66 with a request 72 for a recommendation 70.

The recommendation engine 24 is configured to make an inference 76 thata trigger condition for one or more user recommendation preferences 62,64 will arise, based on the detected current location of the user 14,the behavioral pattern 74 of the user, and one or more contextualfactors 78 associated with current observed user activity. The triggercondition may be one or a set of defined conditions that are specifiedby the user directly or which are determined by the monitoring engine.As some examples, the trigger condition may comprise a predicted futurelocation or other predicted condition of the user 14.

In another example, the recommendation engine 24 may be configured touse machine learning procedures for building predictive models forforthcoming locations of the user and the association of those locationswith the user (home, office, etc.), as well as to learn preferences fromthe data and to identify opportunities occurring in the future in whichthe user may be interested. In this manner, data related to aspects ofusers and users' behaviors and relationships (including graphicalrelationships in a social graph that links users with differentpreferences, attributes, and behaviors), may be collected and leveragedas training and testing data for building predictive models of varioustypes. Tools for building such models include machine learningprocedures such as, for example, Bayesian structure search, SupportVector Machines, Gaussian Processes, logistic regression, and extensionsto relational variants that take into consideration constraints orpatterns of relationships among entities and/or properties. Examples ofpredictions that may be enabled by such predictive models, include auser's or group of users' preferences, future locations of a user (orpresent location if not observed directly), future opportunities inwhich a user may be interested, and user actions in the world.

In one example, a predictive model may be used to generate proposals tothe user for future events that are opportunities for scheduling, andthat may be coupled with commerce and advertising offers. In one case, apredictive model may identify from the calendar program 42 that nextSaturday evening is available for the user. The predictive model maythen generate a wonderful multistep plan for the user and his or herspouse for next Saturday evening. The plan may include, for example, adrive to a location, coupled with one or more activities, such as dinnerand entertainment. The predictive model and recommendation engine 24 mayalso weave together one or more recommendations, offers and/or specialsrelated to the activities and destinations. One or more of therecommendations, offers and/or specials received by the recommendationengine 24 from the recommendation server 66 may be generated accordingto one or more user recommendation preferences and/or the behavioralpattern of the user. In this manner, it will also be appreciated thatthe user activity, behavioral patterns, and other information gatheredby the personal agent program 10 may be used for targeted marketingand/or advertising purposes, provided the appropriate authorizations arereceived from the user.

In addition to providing recommendations and as noted above, therecommendation engine 24 may also take one or more actions on behalf ofthe user with respect to an opportunity occurring in the future. Forexample, in the multistep plan for Saturday evening described above, therecommendation engine 24 may proactively make a dinner reservation forthe user and his or her spouse at a restaurant near one of theirproposed destinations. A message including a recommendation of therestaurant and information related to the reservation may be displayedto the user on the user computing device 12, and/or may be stored forlater access via another computer program, such as the calendar program42. In another example, the personal agent program 10 may proactivelycommunicate with a third party service that desires to deliver anadvertisement to the user in return for an incentive. In this case, thepersonal agent program 10 may receive and store the advertisement andthe incentive, and may inform the user that it has communicated with thethird party service and has downloaded the advertisement/incentive, andis ready to play the advertisement whenever the user desires.

The one or more contextual factors 78 associated with a current observeduser activity describe the context in which user actions in the useractivity take place. The contextual factors may include, but are notlimited to, a date, a day of a week, a time of day, or a time periodthat the user computing device 12 has been located in a detected currentlocation. These and other concepts will be more fully illustrated in theuse case examples that follow.

Turning now to FIG. 2, a manner in which inferences are made andrecommendation requests are generated by the recommendation engine 24will now be described. It will be appreciated that user activity 85output from various computer programs 84, such programs 40-48, 58, and60 described above in relation to FIG. 1, is saved in database 83 ofpersonal agent program 10. User activity 85 includes a stream of currentobserved user activity 86, which is periodically added to a useractivity history 87. The user activity history 87 is reviewed by themonitoring engine 22, described above. The monitoring engine 22 learnsuser behavioral patterns 74 for the user, which are also stored indatabase 83. Aggregate behavioral patterns 88 based on user activity ofan entire user population may be downloaded from the recommendationserver, and stored in database 83 as well, and used to identify learneduser behavioral patterns 74, as described above. Database 83 also storesuser recommendation preferences 62, 64 and their associated triggerconditions 65, which have been directly received as user input via setupmodule 20, or which have been inferred by from user activity 85 bymonitoring engine 22.

The recommendation engine 24 receives at least a portion of the useractivity 85, typically the current observed user activity 86 including acurrent detected location 90 of the user and contextual factors 78, suchas date and time, associated with the current observed user activity.The recommendation engine 24 compares these data to behavioral patterns74, 88 to determine whether a trigger condition 65 of the userrecommendation preferences 62, 64 is likely to be met, for example,within a threshold of probability. If so, the recommendation enginemakes an inference 76 that a trigger condition 65 for the userrecommendation preference 62, 64 will arise.

With further reference back to FIG. 1, upon generation of the inference76, the recommendation engine 24 is configured to send a request 72 tothe recommendation server 66 for a recommendation for a target productor service according to one or more of the user recommendationpreferences 62, 64, as each user recommendation preference 62, 64typically has at least one target product or service associated with it.The recommendation engine 24 is further configured to receive arecommendation 70 related to the target product or service from therecommendation server 66, and display the recommendation 70 in therecommendation GUI 82 on the display 32 of the user computing device 12.

First Use Case Example

In one example use case, user computing device 12 is a mobilecommunication device and user 14 is currently detected to be in Redmond,Wash. via user activity 46′ from the mobile device location trackingprogram 46. User 14 has notified the personal agent program 10 that theuser would like to receive recommendations for highly-rated restaurantsserving Catalan cuisine near Redmond, and for highly-rated restaurantsserving Catalan cuisine in Barcelona, Spain. With reference now to FIG.3, the user has previously inputted these user recommendationpreferences into the user's mobile computing device 12 via a user inputinterface 202 within the recommendation GUI 82.

On another screen of the recommendation GUI 82, the user has alsoprovided the personal agent program with authorization to monitor theuser's calendar activity 42′ in the calendar program 42, location viathe location activity 46′ in the mobile device location tracking program46, browser activity 48′ in the browser 48, and social networkingactivity 60′ in a social networking program 60 that may reside onanother user computing device 54. The user has also inputted userprivacy settings indicating that the user's calendar activity 42′ failswithin an examine-only category that may not be shared externally, andthat the user's location activity 46′, browser activity 48′, and socialnetworking activity 60′ fall within a sharing-authorized category thatmay be sent to a recommendation server with a request for arecommendation.

By monitoring the locations from the user's GPS-enabled mobilecommunication device and mobile device location tracking program 46, anduser activity 42′ from the calendar program 42, including user's sharedcalendar called “FAMILY CALENDAR”, the personal agent program 10 haslearned a behavioral pattern that the user has taken a 2 week familyvacation each August in each of the last 3 years. It is now July, andthe user has a shared calendar item from the FAMILY CALENDAR for August2-August 16 that reads simply “BARCELONA.” Additionally, by monitoringuser activity 48′ from the browser 48, the personal agent program 10learns that the user recently purchased a “LEARN SPANISH” audio bookfrom an online book provider. Based on these contextual factors, thepersonal agent program 10 makes an inference that the user is againplanning a family trip in August, this time to Barcelona, Spain fromAugust 2-August 16. The recommendation engine 24 in the personal agentprogram 10 may also create an additional user recommendation preference62 based on this behavioral pattern, such as a preference forrecommendations of international home swapping services.

By examining the user's social networking activity 60′ and the user'sassociated social graph, the personal agent program 10 notices a postingthat says “Can't wait for Barcelona trip in August” from Friend A, oneof the user's friends who has a residence that is near the user's homein Redmond, Wash. Given this posting, the user's current location inRedmond, the current date, and the user's presumed vacation toBarcelona, the personal agent program 10 makes an inference that atrigger condition for a user recommendation preference for highly-rated.Catalan restaurants near Redmond may arise; namely, that the user mayenjoy meeting Friend A for a meal at a Catalan restaurant near Redmondbefore August 2 to discuss their upcoming travels to Barcelona. Thepersonal agent program 10 may also make another inference that the usermay enjoy dining with Friend A at a Catalan restaurant in Barcelona,should they happen to be in the city at the same time.

The personal agent program 10 sends requests to a recommendation server66 for recommendations for highly-rated restaurants serving Catalancuisine near Redmond, and for highly-rated restaurants serving Catalancuisine in Barcelona. The recommendations received from therecommendation server 66 are displayed in a recommendation region 204 ofthe recommendation GUI 82.

Second Use Case Example

In another example use case, user computing device 12 is a mobilecommunication device and user 14 is currently detected to be at alocation corresponding to the Truck Stop Diner near Knoxville, Tenn.along Interstate 40, via user activity 46′ from the mobile devicelocation tracking program 46. User 14 has notified the personal agentprogram 10 that the user would like to receive recommendations forcoffee shops serving above-average coffee along I-40 between Wilmington,N.C. and Barstow, Calif. With reference now to FIG. 4, the user haspreviously inputted this user recommendation preference into the user'smobile communication device via a user input interface 302 within therecommendation GUI 82.

On another screen of the recommendation GUI 82, the user has alsoprovided the personal agent program with authorization to monitor theuser's calendar activity 42′ in the calendar program 42, location viathe location activity 46′ in the mobile device location tracking program46, browser activity 48′ in the browser 48, email activity 40′ in theemail program 40, phone call activity 44′ in a telephone/messagingprogram 44, and map activity 58′ in a map program 58 that resides onanother user computing device 52, such as a navigation system. The userhas also inputted user privacy settings indicating that the user's emailactivity 40′ and phone call activity 44′ fall within an examine-onlycategory that may not be shared externally, and that the user's calendaractivity 42′, location activity 46′, browser activity 48′, and mapactivity 58′ fall within a sharing-authorized category that may be sentto a recommendation server with a request for a recommendation.

By monitoring the locations from the user's GPS-enabled mobilecommunication device and mobile device location tracking program 46, thepersonal agent program 10 learns that the user began driving 8 hours agofrom the user's residence in Myrtle Grove, N.C. and has been travelingwest on Interstate 40. The personal agent program 10 also notices ashared calendar item dated today on the user's calendar that reads “L.A.TRIP,” Additionally, 8 hours ago the user requested a routing fromMyrtle Grove, N.C. to Los Angeles, Calif. from the map program 58 on thenavigation system. Based on these contextual factors, the personal agentprogram 10 makes an inference that the user is driving from MyrtleGrove, N.C. to Los Angeles along I-40.

The personal agent program notes that the current time is 12:52 pm, theuser has just begun driving west on I-40, and the user's locationremained at the Truck Stop Diner for the previous 47 minutes. Given theuser's presence at this restaurant for 47 minutes over the lunch hour,suggesting that the user has just eaten lunch, and the inference thatthe user will continue driving west on I-40, the personal agent program10 makes another inference that a trigger condition for the user'srecommendation preference for excellent coffee along I-40 may arise;namely, that the user may enjoy stopping for coffee in approximately 1hour and 15 minutes, which corresponds to a predicted future locationthat is approximately 83 miles from the user's current location based onthe user's average driving speed on I-40 during this trip. In making theinference that the user may enjoy stopping for coffee near thislocation, the personal agent program 10 may also utilize related machinelearnings of user behaviors under a variety of conditions over an entireuser population. These machine learnings suggest that users travelingalong freeways on average stop for a coffee or rest break 1 hour and 20minutes after eating lunch.

The personal agent program 10 sends requests to a recommendation server66 for recommendations for coffee shops serving above-average coffeealong I-40 and preferably approximately 83 miles from the user's presentlocation. The recommendation server returns a recommendation for CoffeeShop A in Monterey, Tenn. Monterey, Tenn. is approximately 88 miles fromthe user's current location. The recommendation received from therecommendation server 66 is displayed in a recommendation region 304 ofthe recommendation GUT 82.

The personal agent program 10 may also apply a rule that provides asuggestion to the user that the user take a rest or coffee break whenthe user has been driving on a freeway without a stop for at least 2hours. In the present example, if the user does not stop at Coffee ShopA and is still driving 2 hours after their lunch break, the personalagent program 10 may send a request to the recommendation server 66 forrecommendations for coffee shops serving above-average coffee near thecurrent location of the user or the user's expected route on I-40. Therule may be preset in the personal agent program 10 or may be input bythe user.

Third Use Case Example

In another example use case, and with reference to FIG. 5, usercomputing device 12 is a mobile communication device and user 14 is in acar 350 that is traveling in a directional trajectory 352. User 14 hasnotified the personal agent program 10 that the user would like toreceive recommendations for coffee shops. The user has also provided thepersonal agent program 10 with authorization to monitor the user'scalendar activity 42′ in the calendar program 42, location via thelocation activity 46′ in the mobile device location tracking program 46,and social networking activity 60′ in the social networking program 60that resides on another user computing device 54. By examining theuser's social networking activity 60′, the personal agent programdetermines that the user 14 has a friend 15 with whom the userfrequently meets for drinks or food.

The user's friend 15 is in a car 354 that is traveling in a directionaltrajectory 356. Friend 15 is carrying her friend computing device 55which is also a mobile communication device. Friend 15 has alsoauthorized the personal agent program 10 to monitor her friend activity57 in a calendar program, mobile device location tracking program, andsocial networking program on her friend computing device 55.

By monitoring the locations from the user's mobile communication device,the personal agent program 10 determines that the user is traveling indirectional trajectory 352. Similarly, by monitoring the locations fromthe friend's mobile communication device, the personal agent program 10determines that the friend is traveling in directional trajectory 354.The personal agent program 10 extrapolates from the directionaltrajectories 352, 354 and determines that the directional trajectorieswill intersect at an intersecting location 360. The personal agentprogram also estimates that the user 14 in car 350 will arrive at theintersecting location 360 at approximately 12:42 pm, and the friend 15in car 354 will arrive at the intersecting location at approximately12:44 pm.

The personal agent program checks the calendar program 42 of the user 14and the calendar program of the friend 15 to see if the user and friendare available within a time period that includes the approximate timethat the user and friend will arrive at the intersecting location 360.In the present example, the time period is 15 minutes, it will beappreciated that other time periods may be used, such as 5 minutes, 30minutes, 1 hour or any other suitable time period.

Based on the information determined above, the personal agent program 10sends requests to the recommendation server 66 for recommendations forcoffee shops within a threshold distance of the intersecting location360, such as one block. Other threshold distances may also be used, suchas 3 blocks, 10 blocks or other suitable distances. The recommendationserver returns a recommendation to user 14 that the user and friend 15meet at Coffee Shop B 362 that is located one half block from theintersecting location 360. The recommendation may notify the user 14that friend 15 is expected to be at intersecting location 360 atapproximately 12:44 pm, or 2 minutes after the user is expected toarrive at the intersecting location. The recommendation may also includea coupon, such as a group discount coupon, that provides an incentivefor the user 14 and friend 15 to meet at Coffee Shop B 362. If thefriend 15 has provided the appropriate permissions, the personal agentprogram 10 or recommendation server 66 may send the recommendation tothe friend computing device 55.

With reference now to FIG. 6, a diagram illustrates a method 400 forretrieving recommendations on behalf of a user according to oneembodiment of the present disclosure. The method may be performed usingthe software and hardware components of the personal agent program 10and user computing device 12 described above and shown in FIG. 1, orusing other suitable components.

At 402 the method includes receiving a user authorization to monitoruser activity across a plurality of computer programs used by the useron a user computing device and one or more other user computing devices.As noted above, the plurality of computer programs may include, but arenot limited to, an email program, a calendar program, atelephone/messaging program, a mobile device location tracking program,a browser program, a map program, or a social networking program. Theuser computing device may also be a GPS-enabled mobile computing device.

At 404 the method includes receiving one or more user recommendationpreferences indicating product or service recommendations that the userwould like to receive from recommendation server. At 406 the method mayinclude receiving a user privacy setting indicating an examine-onlycategory of the user activity that the user authorizes the personalagent program to examine from the plurality of computer programs but notshare externally. The user privacy setting may also indicate asharing-authorized category of the user activity that the userauthorizes the personal agent program to send to the recommendationserver with the request for the recommendation.

At 408 the method includes monitoring the user activity with theplurality of computer programs according to the user authorization. Inone example, the user activity may include a detected current locationof the user, in another example, monitoring the user activity mayinclude selectively activating or accessing at least one of theplurality of computer programs based on computing an expected value ofinformation that may be gleaned from the user activity. At 410 themethod includes learning a behavioral, pattern from the user activity.At 412 the method may also include creating an additional userpreference based on the behavioral pattern from the user activity.

Turning now to FIG. 7, at 414 the method may include making an inferencethat a trigger condition for one or more of the user recommendationpreferences will arise, based on the detected current location of theuser, the behavioral pattern of the user, and one or more contextualfactors. The contextual factors may include, but are not limited to, adate, a day of a week, a time of day, or a time period that the usercomputing device has been in the detected current location.

At 416 the method includes sending a request to the recommendationserver for a recommendation for a target product or service according tothe one or more user recommendation preferences. At 418 the methodincludes receiving the recommendation from the recommendation server. At420 the method includes displaying the recommendation on a displayassociated with the user computing device.

Using the systems and methods described above, user activity in avariety of computer programs on one or more computer devices may bepassively monitored to the extent expressly authorized by the user, anduser behavioral patterns may be learned therefrom. Based on thesebehavioral patterns, recommendations may be conveniently retrieved forproducts and services in which the user has expressed a preference, orin which such a preference has been inferred. In this manner, the needsand desires of the user may be proactively anticipated by the systemsand methods described herein.

Regarding the software and hardware operating environments describedherein, it will be appreciated that the terms “module,” “program,” and“engine” have been used to describe software components that areimplemented by processors of the various computing hardware devicesdescribed herein, to perform one or more particular functions. The terms“module,” “program,” and “engine” are meant to encompass individual orgroups of executable files, data files, libraries, drivers, scripts,database records, etc.

It will also be understood that the term “user computing device” mayinclude personal computers, laptop devices, mobile communicationdevices, tablet computers, home entertainment computers, gaming devices,smart phones, or various other computing devices. Further, the processorand memory may be integrated in a common integrated circuitry, as aso-called system on a chip in some embodiments, and the mass storage maybe a variety of non-volatile storage devices, such as a hard drive,firmware, read only memory (ROW, electronically erasable programmableread only memory (EEPROM), FLASH memory, optical drive, etc. Media maybe provided for these computing devices, which contains storedinstructions that when executed by these computing devices causes thedevices to implement the methods described herein. These media mayinclude CD-ROMS, DVD-ROMS, and other media.

It is to be understood that the example embodiments, configurationsand/or approaches described herein are exemplary in nature, and thatthese specific embodiments or examples are not to be considered in alimiting sense, because numerous variations are possible. The specificroutines or methods described herein may represent one or more of anynumber of processing strategies. As such, various acts illustrated maybe performed in the sequence illustrated, in other sequences, inparallel, or in some cases omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnonobvious combinations and subcombinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

The invention claimed is:
 1. A user computing device for retrievingrecommendations on behalf of a user, comprising: a processor of the usercomputing device executing a personal agent program, the personal agentprogram comprising: a monitoring engine executed by the processor of theuser computing device and configured to: monitor user activity with aplurality of computer programs according to a user authorization, theuser activity including a detected current location of and direction oftravel of the user and calendar activity of the user, the monitoringengine being configured to learn a behavioral pattern from the useractivity; and receive notifications of monitored friend activity of afriend using a friend computing device, the friend activity including adetected current location of and direction of travel of the friend andcalendar activity of the friend; and a recommendation engine executed bythe processor of the user computing device and configured to: estimatethat the user and the friend will approach an intersecting locationwithin a predetermined window of time, based on the respective detectedcurrent locations and directions of travel of the user and the friend;determine that the user and the friend are available to meet in thepredetermined time window based on calendar information; send a requestto a recommendation server for a recommendation for a target serviceoffered at a service location within a threshold distance of theintersecting location, which can facilitate a meeting between the userand the friend; receive the recommendation from the recommendationserver, the recommendation including a recommended service offered atthe service location; and display the recommendation to meet the friendat the service location on a display associated with the user computingdevice.
 2. The user computing device of claim 1, wherein a user privacysetting specifies a sharing-authorized category of the user activitythat the user authorizes the personal agent program to send to therecommendation server with the request for the recommendation.
 3. Theuser computing device of claim 1, wherein: the monitoring engine isconfigured to: determine that the user and a friend are traveling indirectional trajectories that will intersect at an intersectinglocation.
 4. The user computing device of claim 1, wherein the pluralityof programs includes an email program, a calendar program, atelephone/messaging program, a mobile device location tracking program,a browser program, a map program, or a social networking program.
 5. Theuser computing device of claim 1, wherein the monitoring engine isfurther configured to dynamically monitor the user activity byselectively activating or accessing at least one of the plurality ofcomputer programs based on computing an expected value of informationthat is gleaned from the user activity.
 6. The user computing device ofclaim 1, wherein the monitoring engine is configured to learn thebehavioral pattern of the user by receiving aggregate behavioralpatterns from the recommendation server, and examining the user activityin the computer programs for user activity that matches the aggregatebehavioral patterns within a threshold degree.
 7. The user computingdevice of claim 1, wherein the monitoring engine is configured to createa user recommendation preference based on the behavioral pattern fromthe user activity.
 8. A method for retrieving recommendations on behalfof a user, comprising: displaying a recommendation graphical userinterface on a display of a user computing device; monitoring useractivity with a plurality of computer programs according to a userauthorization, the user activity including a detected current locationof and direction of travel of the user and calendar activity of theuser; learning a behavioral pattern from the user activity; receivingnotifications of monitored friend activity of a friend using a friendcomputing device, the friend activity including a detected currentlocation of and direction of travel of the friend and calendar activityof the friend; estimating that the user and the friend will approach anintersecting location within a predetermined window of time, based onthe respective detected current locations and directions of travel ofthe user and the friend; determine that the user and the friend areavailable to meet in the predetermined time window based on calendarinformation; sending a request to a recommendation server for arecommendation for a target service offered at a service location withina threshold distance of the intersecting location, which can facilitatea meeting between the user and the friend; receiving the recommendationfrom the recommendation server, the recommendation including arecommended service offered at the service location; and displaying therecommendation to meet the friend at the service location on a displayassociated with the user computing device.
 9. The method of claim 8,wherein a user privacy setting specifies a sharing-authorized categoryof the user activity that the user authorizes the personal agent programto send to the recommendation server with the request for therecommendation.
 10. The method of claim 8, wherein the user computingdevice is a location-aware mobile computing device.
 11. The method ofclaim 8, wherein the plurality of programs includes an email program, acalendar program, a telephone/messaging program, a mobile devicelocation tracking program, a browser program, a map program, or a socialnetworking program.
 12. The method of claim 8, wherein monitoring theuser activity with the plurality of computer programs further comprisesselectively activating or accessing at least one of the plurality ofcomputer programs based on computing an expected value of informationthat is gleaned from the user activity.
 13. The method of claim 8,wherein learning a behavioral pattern includes receiving aggregatebehavioral patterns from the recommendation server, and examining theuser activity in the computer programs for user activity that matchesthe aggregate behavioral patterns within a threshold degree.
 14. Themethod of claim 8, further comprising creating a user recommendationpreference based on the behavioral pattern from the user activity.
 15. Auser computing device, comprising: a processor of the user computingdevice executing a personal agent program, the personal agent programcomprising: a monitoring engine executed by the processor of the usercomputing device and configured to: monitor user activity with aplurality of computer programs according to a user authorization, theuser activity including a detected current location of and direction oftravel of the user and calendar activity of the user, the monitoringengine being configured to learn a behavioral pattern from the useractivity; and receive notifications of monitored friend activity of afriend using a friend computing device, the friend activity including adetected current location of and direction of travel of the friend andcalendar activity of the friend; and a recommendation engine executed bythe processor of the user computing device and configured to: estimatethat the user and the friend will approach an intersecting locationwithin a predetermined window of time, based on the respective detectedcurrent locations and directions of travel of the user and the friend:determine that the user and the friend are available to meet in thepredetermined time window based on calendar information; send a requestto a recommendation server for a recommendation for a target serviceoffered at a service location within a threshold distance of theintersecting location, which can facilitate a meeting between the userand the friend; receive the recommendation from the recommendationserver, the recommendation including a recommended service offered atthe service location, the recommendation generated according to one ormore of one or more user recommendation preferences and the behavioralpattern of the user; take an action on behalf of the user with respectto an opportunity occurring in the future; and display therecommendation and information related to the opportunity on a displayassociated with the user computing device.